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Helmet-mounted  display  targeting  symbology  color  coding: 
An  air-to-air  scenario  evaluation 

Eric  E.  Geiselman  and  David  L.  Post 
Air  Force  Research  Laboratory,  WPAFB  OH  45433-7022 


ABSTRACT 

Laboratory  and  flight  test  evaluations  have  consistently  demonstrated  the  potential  for  dJsp^y 

presented  information  to  enhance  air  combat  performance.  The  Air  Force  Research  Laboratory  s  (AFRL  s)  Helmet-Mou  d 
Sight  Plus  (HMS+)  program  seeks  to  provide  further  enhancement  by  enabling  the  presentation  of  multi-color  symbology  and 
sensor  imagery.  To  take  proper  advantage  of  color-capable  HMDs,  systematic  evaluations  must  be  conducted  to  identify  the 
best  color-coding  techniques.  The  experiment  described  here  is  the  second  we  have  conducted  to  address  this  need.  TTie  firs 
experiment  identified  the  better  of  two  competing  color  coding  strategies  for  air-to-air  weapons  symbology  and  indica  e  a 
pilots  preferred  the  color  codes  over  an  otherwise  equivalent  monochrome  baseline.  The  present  experiment  compared  e 
“winning”  color  code  to  the  monochrome  baseline  during  trials  of  a  complex  multi-player  air-to-air  weapon  delivery  scenano 
Twelve  fighter  pilots  representing  three  different  countries  (U.S.,  U.K.,  and  Sweden)  flew  simulator  trials  that  mcluded  target 
identification,  intercept,  attack,  missile  launch,  and  defensive  maneuvering  tasks.  Participants '  subjective  feedback  and 
performance  data  indicated  a  preference  for  color  coded  symbology. 

Keywords:  air-to-air,  airborne,  cod?~  color,  combat,  display,  helmet,  HMD,  symbology,  targeting. 


1.  INTRODUCTION 

The  US  Air  Force  Research  Laboratory’s  Helmet-Mounted  Sight  Plus  (HMS+)  program  is  developing  a  color  helmet- 
mounted  display  (HMD)  to  enhance  information  conveyance  to  the  pilot.  HMS+  is  essentially  a  color-capable  version  o 
Visually-Coupled  Acquisition  and  Targeting  System  (VCATS)  HMD,  which  has  been  developed  mainly  to  sup^rt  missile 
aiming  over  a  large  range  of  pilot  head  positions  and  orientations.  This  aiming  ability  is  a  major  advantage  of  HMDs  o  e 
head-up  displays  g(HUDs),  as  Bams  (1989)  points  out:  “The  target  is  outside  the  HUD  field-of-view  during  most  tactical 
maneuvers  and  offsets.  The  farther  off  the  target  can  be  detected  and  tracked,  the  more  effective  the  intercept  tactics  can  e. 
The  potential  benefits  of  HMD-presented  information  have  been  demonstrated  m  the  laboratory  and  in  flight  tests  . 

A  full-color  capability  may  be  desired  ultimately  for  more  versatile  HMDs,  but  the  HMS+  program  is  developing  a  two- 
primary,  red  +  green  (RG)  HMD  using  a  subtractive-color  active-matnx  liquid-crystal  display  (AMLCD)  as  the  g 
source5  *.  Subtractive-color  AMLCD  technology  has  been  selected  because  it  provides  better  image  quality  than  conven  ion 
additive-color  AMLCDs  in  this  application7.  An  RG  display  can  produce  reds,  greens,  and  all  th^interv®ning  fJ  ^n  _h’ 
oranges  and  yellows)  but  not  whites,  grays,  blues,  purples,  or  cyans.  The  RG  color  repertoire  should 1  be ' 
given  the  limited  intended  application  for  HMS+.  Furthermore,  for  a  subtractive-color  AMLCD,  the  use  of  RG  instead  of  full 
color  reduces  the  manufacturing  cost  by  39%  and  increases  the  display’s  transmittance  two  to  four  times  . 

In  a  previous  study,  we  developed  two  color-coded  versions  of  the  VCATS  weapons  symbology  using  the  RG  color 
repertoire  .  Six  US  Air  Force  fighter  pilots  with  HUD  and  air-to-air  weapon  delivery  experience  evaluated  the  color  codes 
after  testing  them  while  flying  an  air-to-air  scenario  in  a  simulator.  All  six  pilots  preferred  the  color-coded  symbology  to  e 
monochrome  VCATS  baseline.  Furthermore,  and  perhaps  surprisingly,  a  “red  means  shoot”  color-coding  strategy  (w  ic 
involved  a  progression  from  green  to  red  as  an  indication  of  shoot-criteria  satisfaction)  was  preferred  unanimously  to  a  green 
means  go”  strategy  (which  involved  a  red-to-green  progression). 

Although  the  general  merits  of  color  coding— particularly  for  speeding  visual  search— are  well  documented1®-1112,  it  is 
not  obvious  that  color  coding  HMD  air-to-air  weapons  symbology  should  have  measurable  and  important  performance 


Part  of  the  SP1E  Conference  on  Helmet-  and  Head-Mounted  Displays JV 
Orlando,  Florida  •  April  1999  •  SPIE  Vol.  3689  •  0277-786X/99/S10.00 


66 


benefits,  pilot  preference  notwithstanding.  The  purpose  of  the  present  experiment  was  to  determine  whether  any  such  benefits 
can  be  demonstrated  for  the  “red  means  shoot”  code. 


2.  METHOD 

2.1  Participants 

Twelve  volunteer  military  pilots  from  the  US,  UK,  and  Sweden  participated.  Their  mean  flight  time  was  2448  hours.  Ten 
pilots  had  experience  in  HUD-equipped  fighters,  four  reported  in-flight  HMD  experience,  and  nine  had  air-to-air  missile 
launch  experience. 

23  Apparatus 

We  used  a  40-foot  (12-m)  diameter  projection  dome  with  an  F-16  cockpit  mockup  that  included  a  single  throttle,  force- 
type  side  stick  controller,  HUD,  radar  scope,  attitude  indicator,  and  horizontal  situation  indicator.  The  simulation  software 
used  an  F-16  aerodynamic  model.  The  dome  provided  a  150-degree  horizontal  by  70-degree  vertical  visual  scene,  which  was 
produced  by  six  color  CRT  projectors.  The  HMD  was  simulated  by  drawing  the  symbology  on  the  dome,  superimposed 
additively  on  the  outside  scene  by  the  graphics  processor,  within  a  20-degree  circular  field  of  view  (FOV)  that  was  centered 
about  the  participant’s  line  of  sight  (LOS).  The  LOS  was  measured  by  a  magnetic  head  tracker.  The  HMD  was  blanked  when 
the  tracker  indicated  that  the  participant  was  looking  down  at  the  panel  instruments  or  HUD,  to  simulate  the  action  of  a  real 
cockpit  HMD.  A  black-sky  outside- world  scene  was  used,  to  maximize  the  HMD’s  luminance  contrast.  The  luminances  of  the 
red,  green,  and  yellow  symbols  were  0.20, 0.65,  and  0.85  cd/m2,  respectively,  when  they  were  measured  against  the  black  sky. 
Symbol  contrast  ratios  ranged  from  0.5  to  3.7,  depending  on  the  combination  of  symbol  color  and  background. 

2.4  HMD  symbology  and  color  coding 

We  used  the  VCATS  symbology  (see  Figure  1),  which  is  similar  to  existing  HUD  symbology  and  therefore  familiar  for 
most  tactical  pilots.  It  was  green  in  our  monochrome  conditions.  It  includes  several  functionality  groups,  but  only  those 
relating  to  target  location,  tracking,  and  weapons  deployment  were  color  coded  so  we  discuss  only  these  groups  below. 
Geiselman  et  al.  describes  the  symbology  more  completely9.  For  this  experiment,  the  color-coding  approach  was  as  follows: 
Red  indicated  a  weapon-ready-for-launch  condition.  The  more  things  turned  red,  the  closer  the  selected  weapon  was  to  a 
firing  solution.  The  design  logic  was  that  a  weapon-ready  state  is  a  dangerous  condition  for  the  weapon’s  intended  recipient 
(i.e.,  it  is  an  exocentric  threat).  Thus,  red  indicated  both  a  ready  state  and  a  danger  state:  “Use  extreme  caution  because  the 
missile  is  likely  to  hit  what  is  tracking.”  Green  was  the  default  status  color,  as  well  as  an  “out-of-limits”  indication.  Yellow 
indicated  a  state  approaching  either  acceptable  or  unacceptable  limits.  Identification  symbology  was  also  color  coded.  To 
summarize: 

Green  =  Normal  status/out  of  limits  and  friendly  identification. 

Yellow  =  Approaching  limit  and  unknown  identification. 

Red  ss  Criteria  met  (The  more  red,  the  greater  the  probability  of  success)  and  hostile  identification. 

The  constant  green  components  included  the  ownship,  LOS,  and  general  information  groups.  Green  was  chosen  as  the 
baseline  color  because  it  is  the  conventional  HMD  symbology  color.  The  radar  LOS  circle  was  yellow  to  represent  a  ready 
state. 

TD  box  group.  The  target  designator  (TD)  box  symbols  showed  an  extrapolated  primary  designated  target  (PDT)  LOS 
when  the  target  was  within  the  HMD  FOV;  that  is,  within  a  10-degree  radius  of  the  HMD  center  point.  This  symbology,  like 
all  other  targeting-specific  symbology,  was  present  only  when  a  radar  PDT  was  established.  The  box  symbols  were 
superimposed  on  the  target  location  in  the  outside  scene.  The  TD  box  was  replaced  by  a  target  locator  line  whenever  the 
target  was  within  the  sensor  field-of-regard  but  located  beyond  the  HMD  FOV.  Only  one  PDT  could  exist  at  a  time. 
Information  around  the  box  included  a  shape-coded  identification  friend  or  foe  (IFF)  symbol  on  the  left  side.  Above  the  box, 
an  alphanumeric  readout  showed  either  target  recognition  information  (e.g.,  aircraft  model  identification)  or  the  number  of 
degrees  (if  fewer  than  10)  before  the  target  would  reach  the  onboard  radar’s  maximum  coverage  angle.  (This  number  is 
known  commonly  as  “degrees  before  break-lock.”)  Alphanumeric  indications  of  target  range  and  altitude  were  attached  to  the 
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right  side  of  the  box.  The  bottom  side  was  reserved  for  the  appearance  of  a  star-type  shoot-cue  symbol  when  the  launch 
parameters  for  the  current  selected  weapon  were  satisfied. 

The  color  coding  for  the  TD  box  group  varied  between  green  and  red  depending  on  the  missile  launch  solution.  The  TD 
box  was  green  if  the  PDT  was  outside  the  missile’s  maximum  or  minimum  allowable  launch  ranges.  Within  the  maximum 
flight  (Rmaxi)  and  maximum  terminal  phase  maneuvering  (Rmax2)  PDT  ranges  the  TD  box  changed  continuously '  ^  gree" 
to  yellow.  Within  Rmax2,  the  TD  box  snapped  to  red.  These  color  changes  indicated  the  transition  of  die  PDT  from  out-of¬ 
range  to  within  minimal  launch  parameters  and,  finally,  to  within  a  high  probability-of-success  launch  range.  The  TO  box 
remained  red  within  the  Rmax2  and  minimum  launch  (Rmin)  ranges.  Inside  the  Rmin  range  the  TO  box  snapped  back  to 
green  to  indicate  the  return  to  an  out-of-limits  state.  Additionally,  the  mside-Rmin  state  was  indicated  by  the  presentation  o 
green  break  “X”  symbol  (not  shown  in  Figure  1),  drawn  across  the  center  of  the  HMD  FOV.  Presentation  of  the  TD-box-  (and 
locator-line-;  see  below)  associated  shoot  cue  symbol  indicated  a  within-good-launch-parameters  state,  therefore,  the  shoot 
cue  symbol  was  always  red.  The  degrees-before-break-lock  alphanumeric  associated  with  the  TD  box  and  locator  line  was 
always  yellow  because  this  information,  when  it  appeared,  indicated  an  approaching-limits  condition.  Symbols  representing 
friendly  IFFs  were  green,  unknown  targets  were  yellow,  and  hostile  targets  were  red;  this  system  conforms  with  convention 
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Figure  1.  Composite  VCATS  symbol  set  with  explanatory  labels. 

Taraet  locator  line  group  A  target  locator  line  showed  the  continuously  computed  combined  azimuth  and  elevation 
vecJrm  ^e  PDT  LOsS  the  PDT  was  located  outside  the  HMD  FOV.  The  locator  line  and  TO  box  did  not  coexist 
Accordingly,  the  line  also  had  IFF,  degrees  before  break-lock,  and  shoot  cue  symbology  attached  to  it.  The  line  was  anchored 
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at  the  center  of  the  HMD  FOV  and  radiated  out  toward  the  edge  of  the  HMD  FOV.  The  outside  end  of  the  locator  line  was  a 
solid  arrowhead.  The  length  of  the  line  changed  dynamically  in  proportion  to  the  angular  difference  between  the  HMD  LOS 
and  the  PDT  location,  up  to  45  degrees.  Beyond  45  degrees,  the  line  was  drawn  at  its  maximum  length  of  7.8  degrees. 
Between  45  degrees  and  the  point  where  the  target  crossed  within  the  HMD  FOV,  the  locator  line  shortened  to  its  minimum 
length  of  approximately  2  degrees.  A  shape-coded  IFF  symbol  was  attached  to  the  locator  line  arrowhead.  A  circle  indicated  a 
friendly  return,  an  “X”  indicated  an  unidentified  return,  and  a  diamond  indicated  a  hostile  target.  A  degrees-before-break-Iock 
readout  was  displayed  next  to  the  IFF  symbol  if  the  PDT  was  within  10  degrees  of  the  maximum  radar  coverage.  A  shoot  cue 
was  displayed  at  the  locator  line  anchor  when  the  launch  parameters  for  the  selected  weapon  were  satisfied  for  the  PDT.  The 
locator-line’s  color-coding  matched  the  TD  box’s. 

ASE  circle  group.  The  allowable  steering  error  (ASE)  circle  was  a  boresight-referenced  repeater  of  the  same  symbology 
presented  on  the  HUD.  A  comparison  of  the  dynamic  steering  dot  to  the  edge  of  the  ASE  circle  indicated  the  instantaneous 
quality  of  the  missile  launch  relative  to  the  weapon’s  lateral  and  vertical  limitations.  The  radial  aspect  indicator  line 
represented  the  PDT’s  aspect  angle  relative  to  the  ownship  flight  path. 

The  ASE  circle  was  green  but  the  associated  steering  dot  was  color  coded  according  to  its  proximity  to  the  ASE  circle. 
Within  the  circle,  the  steering  dot  was  red  to  indicate  within-allowable-steering-error.  Outside  the  circle,  but  within  two 
degrees  of  it,  the  dot  was  yellow.  Beyond  two  degrees,  the  dot  was  green.  The  radial  target  aspect  line  attached  to  the  ASE 
circle  was  color  coded  depending  on  the  nose  versus  tail  aspect  relationship  between  the  ownship  and  target  velocity  vectors. 
The  line  was  red  from  270  to  90  (across  the  top)  degrees  aspect  angle  to  indicate  that  the  ownship  nose  was  oriented  to  the 
target’s  tail.  A  yellow  line  between  90-110  and  270-250  degrees  indicated  an  approaching  neutral  aspect.  A  green  line 
between  110-250  degrees  indicated  neutral. 

Dynamic  launch  range  group.  This  symbology  showed  the  PDT  range  and  closure  rate  along  the  left  side  of  a  fixed  scale. 
The  right  side  of  the  scale  showed  significant  launch-parameter  ranges  for  the  selected  weapon.  When  the  PDT  was  within  the 
Rmax2  and  Rmin  envelope,  a  TD-box-  or  locator-line-associated  shoot  cue  was  presented.  If  the  PDT  was  between  the  target 
tum-and-run  range  (Rtr)  and  Rmin,  the  shoot  cue  symbol  was  flashed  at  5  Hz.  A  break  “X”  symbol  (not  shown  in  Figure  1) 
was  drawn  across  the  center  of  the  HMD  FOV  whenever  the  PDT  range  was  inside  Rmin. 

The  range  bars  on  the  dynamic  launch  range  scale  were  colored  to  indicate  the  launch  limitations  they  represented.  The 
Ruiaxi  bar  was  yellow  and  the  Rmax2,  Rtr,  and  Rmin  bars  were  red.  The  line  that  connected  the  Rtr  and  Rmin  bars  to  indicate 
the  Rtr  region  was  also  red.  The  dynamic  target  range  and  closure  rate  caret  were  colored  to  show  the  launch  envelope  region 
in  which  the  target  was  located.  The  caret  was  green  beyond  Rmaxj  and  within  Rmin.  Within  the  Rmax \ — Rmax2  region,  the 
caret  changed  from  greemto  yellow.  The  caret  snapped  to  red  as  the  target  crossed  into  the  Rmax2-Rmin  region. 

2.5  Scenario 

The  scenario  was  a  multi-player  air-to-air  engagement  involving  hostile  bombers,  friendly  fighters,  unknown  fighters,  and 
hostile  fighters.  The  gaming  area  was  a  60  x  50-nm  portion  of  the  Defense  Mapping  Agency  southwest  US  database,  and  the 
Truth  or  Consequences  airport  near  Albuquerque,  New  Mexico  was  homeplate.  The  scenario  involved  the  combined  use  of 
head-down  radar  functions  and  HMD-provided  target  location  information  to  perform  target  acquisition,  radar  lock, 
identification,  and  missile  launch.  There  were  four  mission  phases:  combat  air  patrol,  intercept,  attack  (bombers  and  fighters), 
and  egress.  Our  interest  focused  on  the  intercept  and  attack  phases. 

There  were  six  players:  the  primary  cockpit  (ownship),  two  manned  fighters,  an  autonomous  friendly  fighter,  and  two 
autonomous  enemy  bombers.  The  manned  fighters  were  flown  by  experienced  laboratory  personnel  via  auxiliary  simulator 
stations  and  changed  identification  (i.e.,  from  unknown  to  either  friendly  or  hostile)  according  to  the  experimental  design. 
Their  tactics  were  coordinated  by  an  air-to-air  tactics  expert  acting  as  an  Airborne  Warning  and  Control  System  (AWACS) 
controller  (red  controller).  The  primary  cockpit  was  given  AWACS  type  information  by  the  experimenter  (blue  controller). 
The  controllers  viewed  wall-projected  God’s-eye  views  of  the  gaming  area  and  on-demand  status  information  about  each 
aircraft.  The  players  communicated  via  an  intercom  that  allowed  the  experimental  participant  to  hear  only  the  blue  controller 
and  everyone  else  to  hear  all  other  communication. 

For  each  trial,  the  bombers  started  from  one  of  four  randomly  chosen  locations  and  then  tracked  directly  toward 
homeplate,  flying  one  mile  apart  in  a  trail  formation.  Five  waypoints,  connected  to  form  a  closed  course,  were  used  as  fighter 
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start  locations  The  autonomous  friendly  fighter,  which  served  as  a  distracter  for  the  participant  started  from  a  waypoint 
adjacent  (either  left  or  right,  chosen  randomly)  to  the  bomber  start  location  and  then  also  tracked  directly  t°ward  ho^1^; 
The  manned  fighters  started  from  the  waypoint  adjacent  to  the  bomber  start  location  and  opposite  the  friendly  distracter 
location  As  long  as  the  manned  fighters  were  designated  “unknown,”  they  followed  the  closed  course  counter-clockwise  in 
formationfrom  waypoint  to  waypoint  at  15,000  ft  (4572  m)  and  480  kts.  If  their  identificauon  switched  to  friendly  or  hostile, 
thev  maneuvered  according  to  the  red  controller’s  directions.  Hostile  fighters  maneuvered  aggressively  against  the  ownship 
bu/did  not  fire  on  it.  (The  ownship  was  instructed  to  fly  nonetheless  as  if  hostiles  could  fire.)  Friendly  fighters  maneuvered  to 

cause  confusion. 

2.6  Rules  of  engagement 

Starting  from  homeplate  at  15,000  ft  (4572  m)  and  480  kts,  the  ownship  pilot  was  directed  by  ^  blue  ^ntrolkr  via 
radar  vectors  toward  the  two  approaching  bombers  with  instructions  to  identify  and  shoot  them  down.  While  the  ownship 
maneuvered  the  blue  controller  gave  snap-look  location  information  about  other  “umdenufied  targets  in  the  area.  These 
S~do„  the  ownship  radar  when  they  were  within  the  coverage  volume.  The  ownship  pdotcodd  se  lee t  one  PDT 
at  a  time  using  either  the  conventional  radar  display  (which  had  a  40-mile,  or  64-km,  range  and  ±  60-degree  field-of-regard) 
or  the  HMD. 

The  ownship  pilot  was  instructed  to  monitor  the  other  aircrafts’  movements  during  the  bomber  attack.  Once  the  bombers 
were  defeated  die  ownship  concentrated  on  the  secondary  targets.  During  the  bomber-attack  mission  phase,  the  manned 
fighters  followed  the  prescribed  waypoint  route  and  the  ownship  IFF,  which  was  available  only  on  the  HMD,  showed  them  as 
unknowns  Once  they^eached  their  first  waypoint  (this  took  3  to  5  minutes),  their  IFF  indications  could  change.  The  ownship 
rules  of  engagement  depended  on  the  indications.  Friendly  targets  could  be  ignored,  and  if  only  friendlies  remained,  die 
ovLhip  wi  to  egress  toward  homeplate.  If  the  IFF  was  unknown  and  the  target  was  approaching  homeplate,  the  ownship 
was  to  intercept  and  track  it.  If  the  unknown  turned  away  from  homeplate,  the  ownship  was  to  egress.  If  the  IFF  was  o  1  , 
the  ownship  JL  to  shoot  the  target  down.  The  ownship  was  also  to  egress  if  all  hostiles  were  shot  down  or  all  eight  missiles 

were  used.  The  trial  ended  when  egress  commenced. 

2.7  Procedure 

For  training,  each  participant  read  instmetions,  examined  color  figures  of  the  symbology  set,  listened  to  an  oral 
description  of  the  task,  free-flew  for  30  minutes  to  become  familiar  with  the  simulation,  and  then  experienced  several  full- 
scenario  practice  trials.  Questions  were  addressed  as  they  arose,  both  during  training  and  experimental  trials. 

Each  participant  completed  all  four  1-hour  experimental  sessions  in  one  day,  with  rest  pauses  given  as  needed  and  a  lunch 
break  at  th^halfway  point.  Each  session  consisted  of  8  trials,  yielding  a  total  of  32  trials  per  participant.  The  use  of  color- 
coded  HMD  symbology  alternated  across  sessions:  Even-numbered  participants  began  with  color-coded  symbology  and  odd- 
numbered  parficipants8began  with  monochrome.  Thus,  HMD  color  was  a  within-subjects  variable.  The  start  locations  of  the 
bombers  manned^ircraftf  and  distracter  friendly,  and  also  the  manned  aircraft  idenufications  (i.e.,  both  switch  from  unknown 
to  friendly  both  switch  to  hostile,  one  switches  to  friendly  and  the  other  switches  to  hosule,  or  both  remain  unknown)  were 
Ss  for  control  purport,,  but  were  no,  factor,  in  the  subsequent  analyses.  All  sunulatton  dab,  panuneters 

were  recorded  at  5  Hz  and  all  trials  were  videotaped. 

2.7.1  Subjective  feedback 

After  completing  all  trials,  the  pilots  filled  out  a  questionnaire  designed  to  collect  ratings  and  open-ended  comments.  Of 
the  ^  i„“”s“e™  questions  related  to  the  potential  for  the  HMD  and  color  coding  of  HMD  symbology  to  atd  am-to-atr 
targeting  tasks.  Potential  was  rated  using  the  scale  shown  below. 


None 

Little 

Some 

Significant 

_  Great 

1  ^ 

r  2 

3 

4 

5 

The  first  set  of  items  asked  the  pilots  to:  (1)  Rate  the  overall  potential  for  HMD-presented  information  to  enhance 
perfomaL  for  an  air-to-air  task;  (2)  Rate  the  potential  for  HMD-presented  target  information  to  enhance  target  location  and 
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tracking  tasks;  and  (3)  Rate  the  potential  for  HMD-presented  ownship  information  (attitude,  airspeed,  etc.)  to  enhance 
primary  flying  tasks  while  acquiring  and  managing  targets.  In  a  yes/no  general  feedback  format,  pilots  were  asked  if  there  was 
anything  potentially  confusing  about  the  symbology  they  experienced. 

The  second  set  of  items  asked  the  pilots  to  rate  the  potential  for  color  coding  to  aid  air-to-air  targeting  tasks:  (1) 
Compared  to  the  all-green  symbology  set,  rate  the  potential  of  the  color  coded  HMD  symbology  you  saw  to  enhance  overall 
mission  success;  (2)  Rate  the  potential  for  the  color  coding  you  saw  to  enhance  performance  compared  to  the  all-green 
symbology  for  determining  levels  of  weapons  launch  readiness;  and  (3)  Rate  the  potential  for  the  color  coding  you  saw  to 
enhance  performance  compared  to  the  all-green  symbology  for  target  identification  and  IFF  tasks. 

The  pilots  indicated  their  preferences  for  the  three  different  color-coding  approaches  for  the  simulated  mission  by 
responding  to  questions  such  as: 

•  Do  you  feel  that  color  coding  of  HMD  symbology  has  potential  operational  benefit?  (yes/no) 

•  Do  you  feel  that  the  addition  of  color  coding  to  the  HMD  symbology  you  saw  had  an  effect  on  operational  situation 
awareness?  (yes/no) 

•  Do  you  feel  that  the  addition  of  color  coding  to  the  HMD  symbology  you  saw  had  an  effect  on  operational  workload? 
(yes/no) 

•  Do  you  feel  that  the  addition  of  color  coding  to  the  HMD  symbology  you  saw  had  any  other  effects  worth  mentioning? 

Finally,  the  pilots  were  asked  if  the  scenario  was  a  good  one  for  testing  the  color  codes.  If  the  answer  was  no,  they  were 
asked  to  suggest  how  it  could  be  improved. 

2.7.2  Performance  data 

For  each  launch,  we  computed  PDT  time  prior  to  a  launch  and  the  target’s  range  at  the  time  of  the  launch.  These  PDT 
times  showed  pronounced  positive  skew,  so  we  log-transformed  them.  We  sorted  launches  into  three  probability-of-kill  (Pk) 
categories  according  to  the  shoot-cue  indications  that  were  present  at  launch  time:  A  flashing  shoot  cue  denotes  the  highest 
Pk,  a  nonflashing  cue  denotes  medium  Pk,  and  an  absent  cue  denotes  low  Pk.  We  also  differentiated  between  launches  against 
bombers  and  launches  against  hostile  fighters  because  these  target  types  call  for  different  attack  and  maneuvering  tactics.  We 
calculated  a  mean  logio  PDT  time  and  mean  target  range  for  each  participant  x  Pk  x  target-type  x  HMD-color  combination. 
We  dropped  cells  that  had  fewer  than  two  observations.  One-way  ANOVAs,  using  HMD-color  as  the  main  effect,  were  then 
performed  for  each  combination  of  Pk  and  target-type,  using  only  those  participants  who  had  no  missing  cells. 

3.  RESULTS 


3.1  Subjective  findings  (general): 

The  pilots  indicated  that  the  overall  potential  for  HMD-presented  information  to  enhance  performance  for  an  air-to-air 
task  was  great  (mean  rating  =  4.66).  Likewise,  the  potential  for  HMD-presented  target  information  to  enhance  target  location 
and  tracking  tasks  was  reported  as  great  (mean  rating  =  4.48).  HMD-presented  ownship  information  (attitude,  airspeed,  etc.) 
was  rated  as  having  some  potential  to  enhance  primary  flying  tasks  while  acquiring  and  managing  targets  (mean  rating  = 
3.17).  Seven  of  the  12  pilots  reported  that  the  HMD  symbology  was  potentially  confusing. 

Comments:  (regarding  the  potential  for  confusion) 

L  “ Too  much  lag  in  the  tracker.  ” 

2.  “The  HMD  and  HUD  show  critical  information  in  different  ways .  This  is  unacceptable .  ” 

3.  “[This]  ownship  information  is  unusable.  ” 

4 .  “Head  position  information  is  not  only  useless ,  it  is  actually  confusing.  ” 

5.  “[I]  don't  need  helmet  elevation  and  azimuth  [information].  ” 

6.  “Too  many  numbers  for  practical  use.  ” 

7.  “Too  much  information.  ” 
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3.2  Subjective  findings  (color  coded  symbology): 

ComDared  to  the  all-green  symbology,  the  color  coded  HMD  symbology  was  rated  as  having  some  to  significant  potential 
,o  (meanrating  -  3.50).  Ustng  odor  for  detetmintng  levels  o,  weapons  —  » - 

rated  as  having  significant  potential  (mean  rating  =  4.00).  The  use  of  color  to  enhance  performance  for  target  identification 
and  IFF  tasks  was  rated  as  having  between  some  and  significant  potential  (mean  rating  -  3.50). 


Comments: 

1.  “I  would  fire  immediately  in  range  rather  than  have  to  evaluate.  ” 

2.  "The  color  shoot  cue  has  the  most  effect."  .  „ 

3  “If  the  potential  exists  to  field  a  color  HMD,  fielding  a  monochrome  HMD  would  be  tragic. 

4.  “ Color  does  not  save  a  confusing  symbology  but  it  highlights  information  that  otherwise  would  be  lost. 

5.  "I  found  target  ID  much  simpler  with  color.  ” 


3.3  General  comment  and  yes/no  questions. 

All  12  pilots  agreed  that  color-coded  HMD  symbology  has  potential  operational  benefit  Also,  all  12  pilots  indicated  that 
color-coded  HMD  symbology  had  an  effect  on  operational  situation  awareness.  Ten  pilots  indicated  that  color  HMD 
symbology  had  an  effect  on  operational  workload. 


Comments: 

1.  “A  tremendous  amount  of  potential.  It  will  cut  down  on  fratricide." 

2.  “ Color  is  easier  to  understand  but  I’m  not  sure  of  the  impact  on  operational  performance. 

3.  “Helped  define  threat  earlier  than  digital.  ” 

4.  “Marginally  quicker  processing  of  information.” 

5.  “Color  improved  rapid  SA.  ID  and  Launch  parameters  were  immediately  obvious.  ” 

6.  “Used  the  right  way,  color  could  reduce  the  need  for  alphanumerics. 

7  "SA  and  workload  improved  but  red  is  sometimes  hard  to  read. 

8.  " Color  was  effective  in  that  you  knew  WEZ  info  without  interpretation,  but  the  advantage  was  minimal.  In 

a  turning  fight,  that  may  mean  the  difference  between  having  a  shot  or  not. "  (> 

9.  "[Color]  turned  shot  employment  into  more  analog  instantaneous  recognition  of  valid  parameters. 


3.4  Scenario 

Nine  of  the  12  subject  pilots  reported  that  the  scenario  was  good  for  testing  the  color  vs.  monochrome  HMD  symbology 
Three  reported  that  the  scenario  could  be  improved  by  making  the  task  more  complex  and  including  a  complete  complement 

of  ownship  weapons. 

3.5  Target  identification 

We  expected  that  friendly  and  unknown  targets  would  be  fired  on  occasionally  and  these  errors  might  be  less  frequent 
when  the  IFF  symbology  was  color  coded.  There  were  four  launches  against  friendly  targets  and  seven  against  unknowns,  but 
upon  review  of  the  videotapes,  none  were  attributable  to  HMD  color. 

We  also  thought  participants  might  identify  targets  quicker  when  color-coded  IFF  was  available.  To  test  this  hypothesis 
we  calculated  PDT  times  on  friendly  targets  when  a  hostile  was  present-a  situation  that  called  for  identifying  the  friendly 
quickly  and  moving  on  to  the  critical  task  of  shooting  the  hostile.  Most  of  these  PDT  times  were  less  than  10  s,  but  20 
exceeded  20  s  and  were  deleted  as  outliers,  leaving  249  observations  from  which  we  calculated  a  mean  for  each  participant  x 
HMD-color  combination.  Mean  PDT  times  for  monochrome  and  color  IFF  were  6.8  and  6.5  s,  respectively.  A  one-way 
ANOVA  showed  that  this  difference  is  non-significant,  F(l,ll)  =  0.32,  p  =  0.5848. 
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3.6  Launch  solution  performance 


Flashing  shoot  cue.  PDT  time  for  this  case  was  computed  as  the  difference  between  the  time  at  which  the  cue  started 
flashing  and  the  time  at  which  the  launch  occurred.  For  bomber  launches,  7  participants  had  no  missing  cells;  for  hostile 
fighter  launches,  1 1  had  no  missing  cells.  The  mean  logI0  PDT  times  and  mean  target  ranges  for  these  participants  are  shown 
in  Figure  2.  The  ANOVAs  showed  that  HMD  color  had  no  significant  effect  on  PDT  time  for  bombers,  F(l,6)  -  0.03,  p  - 
0  8578  but  color  coding  reduced  PDT  time?  significantly  for  hostile  fighters,  F(l,10)  =  11.08,  p  =  0.0076.  The  difference 
between  the  reverse-transformed  (i.e.,  geometric)  means  for  the  fighters  is  1.24  s.  HMD  color  had  no  significant  effect  on 
target  range  at  launch  time  for  bombers,  F(l,6)  =  0.01,  p  =  0.9431,  or  hostile  fighters,  F(l,10)  =  0.05,  p  =  0.8262. 


□  Monochrome  ®  Color 


Figure  2.  Geometric  me<*n  PDT  times  and  mean  target  ranges  for  launches  against  bombers  and  ^stile  fighters  when  the 

shoot  cue  was  flashing.  Whiskers  show  standard  deviations. 


Nonflashing  shoot  cue.  PDT  time  for  this  case  was  computed  as  the  difference  between  the  time  at  which  the  cije  came  on 
and  the  time  at  which  the  launch  occurred.  For  bomber  launches,  10  participants  had  no  missing  cells;  for  hostile  fighter 
lannrhpc  there  were  only  7  shots  in  the  monochrome  condition  and  14  in  the  color-coded  condition,  so  we  did  not  perform 
ANOVAs  for  fighter  launches.  The  mean  logio  PDT  times  and  mean  target  ranges  for  bombers  are  shown  in  Figure  3.  The 
ANOVAs  showed  that  color  coding  reduced  PDT  time  significantly,  F(l,9)  =  5.85,  p  =  0.0387,  and  increased  the  target  range 
at  launch  time  significantly,  F(l,9)  =  6.02,  p  =  0.0366.  The  differences  between  the  PDT-time  geometric  means  and  target- 
range  means  are  1.58  s  and  1.17  nm,  respectively.  A  1.17-nm  difference  has  no  practical  effect  on  Pk  for  a  non-maneuvenng 
target,  so  although  there  is  presumably  a  causal  relationship  between  the  reduced  PDT  times  and  increased  target  ranges,  we 
conclude  that  the  participants  shot  sooner  without  sacrificing  shot  quality  in  the  color-coded  condition.  The  sparse  data  for 
fighter  launches  show  the  same  trend. 


Bomber 


Bomber 


Figure  3.  Geometric  mean  PDT  times  and  mean  target  ranges  for  launches  against  bombers  when  the  shoot  cue  was  on  but  not 

flashing.  Whiskers  show  standard  deviations. 
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Shoot  cue  off.  There  were  not  enough  launches  fot  this  case  to  permit  reasonable  ANOVAs.  The  data  resemble  those  for 
the  nonflashing  shoot  cue,  though. 


4.  DISCUSSION 

One  purpose  of  this  project  was  to  collect  military  pilots’  opinions  regarding  die  potential  of  color-coded  HMD 
symbology  to  produce  an  operational  benefit,  compared  to  performing  the  same  task  with  monochrome  symbology.  ape 
coding  and  the  functionality  of  the  symbology  was  held  constant  for  purposes  of  comparison.  A  reasonably  complex  and 
realistic  air-to-air  engagement  scenario  was  employed  for  the  evaluation.  Consistent  with  past  findings,  the  pi  o 
overwhelmingly  reported  that  HMD  technology  has  potential  to  aid  the  air-to-air  targeting  task.  The  pilots  also  reporte  a 
the  addition  of  color  furthered  the  performance  enhancing  potential  of  the  HMD.  Also  consistent  with  our  past  findings,  e 
“red  means  shoot”  color  code  was  deemed  effective. 

Our  objective  results  show  that  the  “red  means  shoot”  code,  when  applied  to  the  VCATS  weapons  symbology,  produced 
significandy  faster  shots  against  fighters  and  bombers,  without  degrading  Pk.  For  fighter  targets,  this  advantage  was 
demonstrable  only  for  flashing  shoot  cues  whereas,  for  bomber  targets,  it  was  demonstrable  only  for  nonflashing  shoot  cues^ 
This  difference  almost  certainly  reflects  a  difference  in  tactics:  Military  pilots  know  that  launches  against  agile  fighters  should 
be  delayed  undl  the  best  possible  launch  solution  is  available,  whereas  launches  against  bombers  can  be  successful  under  less 
optimal  conditions.  Furthermore,  in  our  scenario,  it  was  important  to  shoot  the  bombers  quickly  and  move  on  to  the  fighters. 

Our  best  estimates  of  the  average  time  reductions  due  to  color  coding  are  1.24  and  1.58  s  against  fighters  and  bombers, 
respectively,  albeit  for  different  launch  solutions.  These  might  not  seem  like  substantial  differences,  but  in  contemporary  air- 
to-air  combat,  even  fractions  of  a  second  can  have  life  or  death  consequences.  Viewed  from  this  perspective,  die  time  savings 
demonstrated  here  are  impressive,  and  even  if  they  proved  to  be  only  half  as  great  in  the  more  complex  environment  of  real 
combat,  they  would  still  be  sufficient  to  make  color  HMDs  worth  serious  consideration. 

It  is  natural  to  wonder  whether  the  advantage  we  seem  to  have  found  for  color  coding  is  related  to  luminance  contrast 
rather  than  color.  Clearly,  the  contrasts  we  were  able  to  produce  are  mostly  below  the  minima  recommended,  for  example,  for 
visual  display  terminals1314,  and  it  might  be  that  it  was  simply  harder  to  read  the  monochrome  green  symbology.  Examination 
of  our  figures,  however,  shows  that  most  launches  occurred  at  least  several  seconds  after  the  shoot  cue  appeared,  which  seems 
too  long  a  delay  if  the  pilots  were  merely  shooting  as  soon  as  an  appropriate  cue  became  legible.  We  believe  the  advantage  of 
color  coding  was  that  it  eliminated  the  need  to  read  or  even  foveate  the  symbology  at  all,  freeing  the  pilots  to  focus  their 
vision  and  attention  on  other  matters  that  influence  the  decision  to  shoot  and  thus  recognize  a  good  opportunity  quicker. 

# 

Our  next  step  will  be  to  re-design  the  symbology  to  try  to  further  exploit  the  apparent  advantages  of  color  coding.  A 
similar  evaluation  will  be  performed  to  compare  the  new  format  to  the  baseline  color  symbology. 
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